3D Markerless Human Limb Localization through Robust Energy Minimization

Markerless human tracking addresses the problem of estimating human body motion in non-cooperative environments. Computer Vision techniques combined with Pattern Recognition theory serve the purpose of extracting information on human body postures from videosequences, without the need of wearable markers. Multi-camera systems further enhance this kind of application providing frames from multiple viewpoints. This work tackles the application of multi-camera posture estimation through the use of a multi-camera environment, also known as ”smart space”. A 3D skeleton structure and geometrical descriptors of human muscles are fitted to the volumetric data to directly recover 3D information. 3D skeleton deformations and bio-mechanical constraints on joint models are used to provide posture information at each frame. The proposed system does not require any pre-initialization phase and automatically adapt the skeleton and the volumetric occupation of each limb to the actor physiognomy independently from the pose. Exhaustive tests were performed to validate our approach. Gesture tracking and posture estimation require the localization of the human body in the scene and the estimation of each limb position during motion. This process can be considered as a combination of two major components, namely, Model Definition and Pose Estimation. Model definition aims at the moulding of the human body model that best fits to the specific shape and size of the tracked human subject, whereas pose estimation is devoted to the proper model adaptation to the postures and the body joint configurations during the motion evolution [1]. Only a few human motion tracking approaches use images from multiple cameras in order to obtain cues on 3D information. On the other hand, a broad range of applications is based on a single monocular acquisition device,e.g.[2]. Motion tracking solutions can be parsed into three main classes: marker-based, model-based and model-free. Our aim is the realization of a markerless system able to estimate the human posture evolution exploiting visual features such as colors, edges, silhouettes and textures. The output of such a system is a set of angles relating each limb to each other, submitted to appropriate constraints for each joint (articulations differ in possible rotation angles and angular extensions). Various approaches that explicitly model the human body as an assembly of rigid parts can be found in the literature[3].Most of them adopt a two-stage strategy: a bottom-up detector is first applied on the image in order to extract the candidate parts, then, a top-down procedure is used to in ria -0 03 26 75 7, v er si on 1 5 O ct 2 00 8 Author manuscript, published in "Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications M2SFA2 2008, Marseille : France (2008)"